Repository navigation
Sync upstream transformers main into OMY - #10
Merged
Merged
Conversation
* [debug] Poll GitHub API 10x to observe cache inconsistency in schedule runs Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Retry get_daily_ci_runs when GitHub API returns stale cache results The GitHub Actions search index (used for event=/branch= filters) can lag behind the database — different backend nodes return wildly different total_count values (190, 238, 311, 413 observed for the same URL within minutes), and the most-recent runs are missing from stale responses. Detect staleness by checking whether the current GITHUB_RUN_ID appears in the returned list; if absent, retry up to 3 times with a 30s delay. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [debug] Use get_daily_ci_runs in debug job; add current_run_id param Add optional `current_run_id` parameter to `get_daily_ci_runs` so callers can supply a known schedule run ID for stale-cache detection when GITHUB_RUN_ID belongs to a non-schedule-triggered job (e.g. push). Update the debug workflow job to call the Python function directly (two quick back-to-back calls) instead of raw curl, so the retry logic is exercised end-to-end and visible in the CI logs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [debug] Force retry path with fake run ID to verify stale-detection logging Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Revert "[debug] Force retry path with fake run ID to verify stale-detection logging" This reverts commit e49340f. * [debug] Run get_daily_ci_runs 20 times to catch stale cache hit Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [debug] Sleep 60s between get_daily_ci_runs calls to hit different cache nodes Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Increase get_daily_ci_runs max_attempts from 3 to 5 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add unit tests for get_daily_ci_runs stale-cache retry logic - tests/utils/test_get_previous_daily_ci.py: 5 tests covering fresh hit (no retry), stale→fresh (one retry), all stale (max_attempts exhausted), no current_run_id (stale check skipped), and empty schedule fallback to workflow_run event - utils/get_previous_daily_ci.py: ruff formatting fix (long print line) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Infer stale-check eligibility from current run's workflow_id Instead of an explicit `current_run_id` argument, `get_daily_ci_runs` now always fetches the current run's metadata (GITHUB_RUN_ID) upfront and compares its `workflow_id` to the queried one: - Same workflow → stale-cache detection applies (max_attempts=5, 30s sleep) - Different workflow → skip (e.g. AMD CI querying Nvidia CI runs); left for a follow-up PR once the same-workflow case is confirmed stable Also adds a TODO comment on the event=workflow_run fallback path (AMD CI) and updates the 6 unit tests accordingly (new test for the different-workflow case; all tests account for the upfront current-run lookup call). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Gate stale-cache check on GITHUB_EVENT_NAME == 'schedule' Push- and dispatch-triggered runs won't appear in event=schedule results even when the API is fresh, causing spurious retries. Only enable the stale-cache retry loop when the current run is itself schedule-triggered. Adds a corresponding unit test (test_non_schedule_event_skips_stale_check). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Revert debug changes to self-scheduled-caller.yml Restore the workflow to its main-branch state; the debug polling job was only needed to verify the stale-cache behaviour. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Condense get_daily_ci_runs comments and refer to huggingface#48374 for rationale Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
…ckend (huggingface#48341) * Fix `safe_open` mmap memory exhaustion on Windows by using `pread` backend On Windows, memory-mapping safetensors files reserves copy-on-write commit charge for the entire file. For large multi-shard checkpoints this exhausts virtual memory. Switch to the `pread` backend on `win32` in both `_load_pretrained_model` and `MtpModel` loading paths. * Address review: add MPS backend fix to MtpModel, hoist backend selection before loop, use targeted sys.platform patch in test * Address review: remove tests
…uggingface#48191) * [ONNX] Skip affected models on torch >= 2.13 (two dynamo ONNX regressions) torch 2.13.0 introduced two regressions in dynamo ONNX export: - pytorch/pytorch#194381: aten.sub type-promotion failure for scalar - int_tensor - pytorch/pytorch#194382: aten.mul.Scalar missing ONNX decomposition Skip all 35 affected model classes under `EXPORT_SKIPS["onnx"]` when torch >= 2.13, using the existing skip infrastructure. Will be removed once the upstream PyTorch fixes land. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Scope ONNX skip to torch == 2.13.x only (auto-runs on 2.14+) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add missing BigBird subclasses to torch 2.13 ONNX skip list Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [ONNX] Clarify skip guard is torch == 2.13.x only (not >=) Update comment and skip reason strings to say `torch == 2.13` instead of `torch >= 2.13`, making it clear the guard auto-lifts on 2.14+. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [ONNX] Fix the torch 2.13 scalar regressions instead of skipping the models Both regressions are a Python float meeting an integral tensor, whose promotion torch 2.13 mishandles: `1.0 - int_mask` crashes the decomposition pass (pytorch/pytorch#194381) and `int_mask * 2.0` reaches translation with no registered ONNX decomposition (pytorch/pytorch#194382). On the same torch, `1.0 - float_tensor` and `float_tensor * 2.0` export fine — so promoting the tensor operand up front, to the dtype the op already produces, is enough. The op and its overload are left alone, and the inserted cast carries the op's own `meta` because for these elementwise cases it is the same value. Decomposition also emits `mul.Scalar` with a *symbolic* second operand (a division result, not a literal). There is no constant to promote there, so that one is rewritten to `mul.Tensor`, which has the two-operand translation — the same rewrite `_fix_remainder_scalar` makes for the same reason. Reachable because the FX fixes run again right after `run_decompositions`. This drops the 39 skip entries: the 15 affected families export again on 2.13, including the vision models (Sam, SamHQ, GotOcr2, GroundingDino, SegGpt, EfficientLoFTR, DeepseekOcr2). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Resolve the promoted-op set on first use, not at import `exporter_onnx` is importable without torch — the CI job that imports transformers with PIL only proved this the hard way — and naming `torch.ops.aten.*` overloads in a module-level frozenset broke that with a `NameError` before anything ran. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Say what the `mul.Scalar` operand is, and check it Review raised two things. The `Node` check assumed a tensor operand: it is never one. Across the affected families all 33 sites are an `operator.truediv` result, i.e. a `SymFloat`, which `mul.Tensor`'s translation does take — so the rewrite stands, but the guard now names the forms that op accepts instead of trusting the node type, and anything else keeps the `mul.Scalar` overload and fails visibly in translation. The fixes are also deliberately not version-gated, which the docstring implied they were. Both rewrites are semantics-preserving on any torch — a cast to the dtype the op already produces, and an overload swap with the same meaning — so gating would only decide which torch exercises the path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: IlyasMoutawwakil <moutawwakil.ilyas.tsi@gmail.com> Co-authored-by: Ilyas Moutawwakil <57442720+IlyasMoutawwakil@users.noreply.github.com>
…d output (value drift) (huggingface#48376) Update Qwen3VLMoe batch integration test expected output (value drift) Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
…to avoid MoE disk offload issue (huggingface#48377) * [LongcatFlash] Fix test_longcat_generation_cpu by using device_map="cpu" `device_map="auto"` causes accelerate to offload MoE expert weights to disk, which then fails to reload them due to an internal weight format incompatibility. Since the test already requires large CPU RAM, use `device_map="cpu"` to keep all weights in memory and avoid disk offloading entirely. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [LongcatFlash] Update golden string and skip test_longcat_generation_cpu on small runners - `test_shortcat_generation`: update expected output to current model output (value drift) - `test_longcat_generation_cpu`: replace `@require_large_cpu_ram` with `@require_torch_accelerator_memory(memory=1100)` — the 562B parameter model requires ~1,047 GiB of bfloat16 weights, far exceeding the CI runner budget (84 GiB single / 168 GiB dual), and disk offloading fails due to MoE weight format incompatibility with accelerate Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * remove unused require_large_cpu_ram import Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Syncs the latest
huggingface/transformersmaininto OMY and resolves the merge conflict insrc/transformers/modeling_layers.py.Changes
upstream/maininto OMYmain.src/transformers/modeling_layers.pyby keepingimport pcre as reand adding the newimport sysfrom upstream.src/transformers/models/vibevoice/convert_vibevoice_to_hf.pyto useimport pcre as reso OMY's regex standardization stays consistent after the sync.make stylepasses locally.Link to Devin session: https://app.devin.ai/sessions/92d48d7ab1a64fc98c92fc894cd1dadb
Open in Devin Desktop: https://app.devin.ai/desktop/session/92d48d7ab1a64fc98c92fc894cd1dadb?variant=devin
Requested by: @Qubitium